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In two-flavor dense quark matter, we describe the dynamics in the single plane wave Larkin- 
Ovchinnikov-Fulde-Ferrell (LOFF) state satisfying the color and electric neutrality conditions. We 
find that because the neutral LOFF state itself suffers from a chromomagnetic instability in the 
whole region where it coexists with the (gapped/gapless) two-flavor superconducting (2SC/g2SC) 
phases, it cannot cure this instability in those phases. This is unlike the recently revealed gluonic 
phase which seems to be able to resolve this problem. 
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It is natural to expect that cold quark matter may ex- 
ist in the interior of compact stars. This fact motivated 
intensive studies of this system over the past few years 
(for a review, see Ref. Q). Matter in the bulk of stars 
should be electrically neutral. It should be also a color 
singlet. The electric and color neutrality conditions play 
an important role in the dynamics of quark pairing Isj.yf . 
Several states have been studied in two and three-flavor 
dense QCD aspossible ground states satisfying these con- 
ditions 0SSIEI10. 

Recently, it has been revealed that the 
(gapped/gapless) two-flavor superconducting 

(2SC/g2SC) phase suffers from a chromomagnetic 
instability connected with the presence of imaginary 
(tachyonic) Meissner masses of gluons [|J. Later a 
chromomagnetic instability has been also found in the 
color-flavor locked phase of three- flavor QCD 9]. It is 
clear that the resolution of this problem is intimately 
connected with the determination of the genuine ground 
state in dense quark matter. Therefore it is one of the 
central issues in the field. 

In this Letter, we will study this problem in neu- 
tral two-flavor dense QCD. At present, there are two 
types o f dy namics suggested for its resolving. In Refs. 
EfllHllI, the single plane Larkin— Ovchinnikov-Fulde— 
Ferrell (LOFF)-like dynamics [H E3 were considered. 
[Although the dynamics considered in Refs. [ITl IT^ 
look different from the conventional LOFF one, by us- 
ing an appropriate gauge transformation, one can show 
that they are in fact equivalent to the latter.] The sec- 
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ond type is the dynamics of t he g luonic phase introduced 
and studied recently in Ref. 15]. In that phase, gluonic 
degrees of freedom play a crucial role and the dynamics 
is manifestly non-abelian. It is clear that it would be 
important to discriminate these two types of dynamics 
in order to establish the genuine ground state. It is the 
primary goal of this Letter. 

We recall that in the two-flavor case, the manifesta- 
tions of the chromomagnetic instability are quite differ- 
ent in the regimes with Sfi < A < and A < S/i 
0, where 5/j yields a mismatch between chemical po- 
tentials for the up and down quarks and A is a diquark 
gap. The (strong coupling) regime with S/j, < A corre- 
sponds to the 2SC solution, and the (intermediate cou- 
pling) regime with A < 5ji corresponds to the gapless 
g2SC one 0. While in the g2SC solution both the 4- 
7-th gluons and the 8-th one have tachyonic masses, in 
the 2SC solution, with 8(i < A < \/2Sfj,, only the 4-7-th 
gluons are tachyonic. 

The main result of this Letter is that the neutral LOFF 
state itself suffers from a chromomagnetic instability in 
the whole region where it coexists with the 2SC/g2SC 
states. Because of that, it cannot cure this instability 
in those states. This is unlike the gluonic phase which 
seems to be able to resolve this problem. 

Studying the neutral LOFF state in two-flavor quark 
matter is a nontrivial problem. Up to now, such stud- 
ies have been done only in the weak coupling dynamical 
regime in which neither the neutral 2SC nor neutral g2SC 
solutions exist 16]. Here we will consider the neutral 
LOFF phase both in the strong coupling and intermedi- 
ate coupling regimes that allows to clarify the role of the 
LOFF dynamics for curing the chromomagnetic instabil- 
ity. 

To study dense two-flavor quark matter in 0- 
equilibrium, we consider a phenomenological Nambu- 
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Jona-Lasinio (NJL) model whose Lagrangian is 



where e 1 ^ and e Q ^ 7 are the antisymmetric tensors in the 
flavor and color spaces, respectively (as usual, we neglect 
the current quark masses). In /3-equilibrium, the ele- 
ments of the diagonal chemical potential matrix fi for up 
(u) and down (d) quarks are fi ur = fi ug = fi — Sfi, fi u b — 
fi - Sfi - fig and fi dr = fidg = fi + Sfi, fi d b = fi + Sfi - fig, 
where fi = fi— Sfi/3 + fig/3 and Sfi = fi e /2. Here the sub- 
scripts r, g, and b correspond to red, green and blue quark 
colors, fi is the quark chemical potential (the baryon 
chemical potential \ib is given by fiB = 3/i), fi e is the 
chemical potential for the electric charge, and fi% is the 
color chemical potential. The latter is connected with the 
vacuum expectation value (VEV) of the time component 
of the 8-th gluon 

By using the auxiliary field <£>" ~ iip ee a 75'0, the La- 
grangian density Q can be rewritten as 
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-i$ Q [# £ e a 7 5^ C ] - \[i^ C ee a l5 ^* a . (2) 
The 2SC and g2SC states have a nonzero diquark conden- 



sate ($ Q )| a =6 = A chosen along the blue color direction. 
As a result, the color group SU (3) c is spontaneously bro- 
ken to SU (2) c and blue quarks are gapless. 

The order parameter of a single plane wave LOFF 
state has the form = Ae 2 ^ s with a con- 

stant modulus A and a constant vector q in the expo- 
nent. Then it is convenient to perform a gauge trans- 
formation if) — ► ip 1 = exp(—2y/3iq-xT 8 ) tp, where T 8 = 
^4= diag(l, 1, — 2) is the eighth generator of SU C {3) in 
the fundamental representation. This gauge transforma- 
tion trades the x-dependent phase of the LOFF order 
parameter for the ^-independent term — 2a/3 tp"/-qT s ip in 
the kinetic term for quarks. Obviously, the equality 
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(3) 



takes place, i.e., the vector qis proportional to the VEV 
of a spatial component of the eighth gluon. 

To derive the free energy (effective potential) we need 
to calculate the fermion determinant. For this purpose, 
it is convenient to introduce the Nambu-Gor'kov spinor 
# T = ($ir,rfi!-,'ipig,'ipig,i>ib,i>ib)- Then, noting that the 
contribution of the gapless blue quarks factorizes and ap- 
plying the standard formulas for 2 by 2 matrices in the 
red and green quarks sector, we find 



d C ts-\p ,p)= n 

T 3 =±l 



(p +fi-SfiT 3 -fi 8 ) 2 ~{p-2qf 



n 

T 3 = ±l 



1 2 



(p -fi + 5fii 3 +fi 8 ) 2 -(p + 2tf 



n 

T 3 =±l 



1 4 

{(p° - SfiT 3 -Q) 2 - (E A ^} {(/ - 5^ + Q) 2 (El q f) + 4A 2 (Q 2 - q 2 ) , 



(4) 



where we defined 

Q = l(\p + q\-W-<i\), q=\q\, (5) 

E L = \l\( E P +q + E P- q ) 2 + A2 > ( 6 ) 

E± +q = \p + q\±fi. (7) 

The free energy is expressed through an integral of 
lndetS 1-1 . Because the vector q'm the LOFF phase is just 
a redundant variable for gapless blue quarks, one can put 
q = in the part of the free energy connected with these 
quarks. Then it is obvious that their contribution to the 



free energy coincides with that of free massless fermions. 
In order to perform the loop integral for the red and 
green quarks part, we make a usual assumption of the 
dominance of the region around the Fermi surface, with 
p = \p\ ~ fi. Then, neglecting contributions suppressed 
by 1/fi 2 , we can omit the last term 4A 2 (Q 2 — q 2 ) in Eq. 
ijljl and use the approximate relations 

Q = qcos9 + o(%] , cos(9=^-^, (8) 
\P J pq 
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where = y/(p± y) 2 + A 2 . Evaluating now the loop 
integral in the hard dense loop approximation, we obtain 

I 



the following expression for the free energy of the two- 
flavor LOFF state: 



Q(fi,6(j,,iJ,g,A,q) = Q.2SC 
with the 2SC part f^sc being 
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where A is a ultraviolet cutoff in the NJL model, zi 

A 2 



<?(1 



1 



A 2 



with the step function 0(a:), 



(S M +<?) 2 ' V (^+9) 2 

and the contribution of electrons was also included in 

^2SC- 

The following remarks are in order. Because we use 
the sharp cutoff A in evaluating the loop integral, it is 
necessary to subtract the corresponding vacuum contri- 
bution with ft = us = i5/i = A — and the same value of 
the vector q: This leads to removing the spurious term 
A 2 <f 2 in the free energy. One can easily check that for 
q = 0, expression (fTUl) yields the correct 2SC/g2SC free 
energy, and for A = 0, q = 0, it reproduces the correct 
free energy for the normal phase [ig. Last but not least, 
as will be discussed below, the one-loop (i.e., mean field) 
approximation we use is reasonable for realistic values of 
the chemical potential /z. 

The equations of motion for the free energy Q10[l yield 
four equations: Two gap equations for A and q and two 
neutrality conditions for 5/i and fi%: 



on 

9A " ° : 



(12) 

We analyzed both numerically and (in some limiting 
cases) analytically these equations. The analysis of Eq. 

(|12k for (j,g shows that it is suppressed as fi$ ~ O (■^r) 

or //§ ~ O Op) that allows to put /ig = in Eqs. 1)11) [I. 

lUla), iJHb), and 

It is expected that the typical values of the quark chem- 
ical potential /i in compact stars should be in the window 
300 — 500 MeV. In the numerical analysis of equations 
IJHJ, we take //=400 MeV and A = 653.3 MeV [as 
will be discussed below, the results are not sensitive to 
the choice of the value of [i in the interval 300 — 500 
MeV] . The NJL coupling constant Ga determines physi- 
cal properties of the model. In practice, it is more conve- 



nient to trade Ga for Ao, which is the value of the 2SC 
gap at (5^=0. Each of Eqs. da), (JUb), and lfl2lc1 with 
/ig = determines a surface in three dimensional space 
with coordinates A, q, and fi e — 2dfi. The intersection of 
the neutrality surface, determined by Eq. (|12l c). and the 
surface determined by gap equation 1)1 2\ b) yields a neu- 
trality curve. Its projections on the (/x e ,A) and (n e ,q) 
planes are shown in Figs. H(a) andn(b), respectively. 
For a fixed Ao, the solution for a neutral LOFF state is 
depicted by a black bold point which is an intersection 
of the neutrality curve and the "gap" curve determined 
by two gap equations 112l a1 and (|12l b'l. In Fig^ it is 
shown the neutrality curve of the LOFF state (bold line) 
and three gap curves (thin lines) for three characteristic 
values of Ao: 40 MeV (weak coupling), 110 MeV (inter- 
mediate coupling), and 190 MeV (strong coupling). The 
neutral LOFF solution occurs only for the intermediate 
coupling, corresponding to the gapless g2SC solution. 

The analysis of Eas. (|12l aV (|12l c) shows that the neutral 
LOFF solution exists in the window 

63 MeV < A < 137 MeV (LOFF window) , (13) 

which can be compared with the g2SC window 

92 MeV < A < 130 MeV (g2SC window) . (14) 

We emphasize that for the same value Ao (i.e., the same 
value of the coupling Ga), the values of the parameters 
A and 8[i are different for the neutral 2SC/g2SC and 
neutral LOFF phases. 

In the LOFF window H2J), we find that A/A = 
0-0.83. The calculated free energies in different phases 
are shown in Fig. [21 (the free ener gy o f the normal phase 
was chosen as a reference point) 19]. One can see that 
the neutral LOFF phase is more stable than the neu- 
tral normal phase in the whole LOFF window H13fl where 
these phases coexist. It is also more stable than the neu- 
tral g2SC/2SC phase in the whole g2SC window (JTU) plus 
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FIG. 1: The neutrality curve of the LOFF state (bold line) 
and solutions of the gap equations (thin solid lines) for three 
different values of Ao. The dashed curve in (a) is the neutral- 
ity line for the 2SC/g2SC state. The dash-dotted line in (a) 
is the A = 5/j, line. 



the narrow region 130 MeV < A < 136 MeV in the 2SC 
phase near its edge with the g2SC one. However, because 
the chromomagnetic instability in the 2SC phase occurs 
at 5[i — A/y/2, which corresponds to Ao = 177 MeV, 
the neutral LOFF solution cannot cure this instability in 
a wide region, with 136 MeV < A < 177 MeV, in that 
phase. 

It is however not the end of the story. Combining Eq. 
(|12[) with Eq. (68) for Meissner masses in the second 
paper in Ref. lfj, we found that the Meissner masses 
for the 4-7-th gluons in neutral LOFF state are imag- 
inary for all values of the parameter Ao > A" =81 
MeV, see Fig. [3] (the Meissner mass of the 8-th gluon 
is zero 10]). In other words, for these values of Ao, the 
neutral LOFF state itself suffers from a chromomagnetic 
instability. Because the values of Ao both in the 2SC 
phase and in the g2SC window l|14fl are larger than this 
critical value Ag r = 81 MeV (see Fig. |2J), we conclude 
that in the whole region where the neutral 2SC/g2SC 
and LOFF phases coexist, all these three phases are un- 



10 




40 60 80 100 120 140 160 180 200 
A (MeV) 



FIG. 2: Free energy of the neutral LOFF state (bold line), 
the neutral 2SC state (solid thin line), and the g2SC state 
(dashed line). The free energy of the normal state is chosen 
as a reference point. 



stable. Therefore, the neutral LOFF state with a single 
plane wave cannot cure the chromomagnetic instability. 

We checked that this conclusion is not sensitive to 
the choice of the value of the quark chemical potential 
in the interval 300 MeV < fi < 500 MeV. In particu- 
lar, we found that for fi= 300 MeV, the LOFF window 
is 49 MeV < A < 121 MeV and the g2SC window is 
72 MeV < A < 115 MeV. The neutral LOFF state has 
the chromomagnetic instability for Ao > Ag r = 64 MeV 
in this case. As to fi= 500 MeV, the LOFF and g2SC 
windows are 74 MeV < A < 141 MeV and 109 MeV < 
A < 134 MeV, respectively. The neutral LOFF state 
has the chromomagnetic instability for Ao > Ag r = 94 
MeV. Thus, the conclusions of the present analysis seem 
to be robust. 

As to the reliability of the mean field (one-loop) ap- 
proximation we use in this problem, Fig. shows that 
the values of the ratio A/fi in the neutral LOFF state 
are typically less than one fourth. This suggests that the 
mean field approximation should be at least qualitatively 
reliable. Still, it would be certainly important to clarify 
more explicitly the role of fluctuations in this problem. 

Because of the relation between the LOFF vector q and 
the VEV of A^ in Eq.©, the chromomagnetic insta- 
bility in the neutral LOFF state implies that the vector 
condensate ( ) alone is not enough for washing out the 
chromomagnetic instability. We consider this as a strong 
indication on the relevance of the gluonic phase |15J for 
curing this problem. Indeed, in the gluonic phase there 
are additional vector condensates directly connected with 
the 4-7-th gluons and, as was shown in Ref. [l5|. there is 
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FIG. 3: Meissner mass square M 2 for the 4-7-th gluons in 
the neutral LOFF state. The QCD coupling constant a 3 — 
g 2 /(47r) is chosen to be equal to 1. 



no chromomagnetic instability in that phase in the strong 
coupling regime close to the value Ao = 177 MeV corre- 
sponding to the end point S/i = A/y2 of this instability 
in the 2SC solution. 



This makes the study of the gluonic phase both far- 
away from the edge A = 177 MeV of the chromomag- 
netic instability in the 2SC solution and in the g2SC win- 
dow to be quite desirable. Also, it is clear that it would 
be interesting to study the neutral LOFF state in the 
three-flavor QCD |2J| and to examine the relevance of 
the LOFF state with more than one plane waves [2l| for 
curing the chromomagnetic instability. 

The work was supported by the Natural Sciences and 
Engineering Research Council of Canada. 
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